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OVERVIEW 

By now, most libraries involved to any 
extent with Total Quality Management 
realize that successfully gathering 
information from their customers is the key 
^ to improving services. But how does one go 
^ about identif)dng the needs of customers 
Q with competing demands and different 

perceptions? Our answer is that libraries 
have to employ a variety of techniques. The 
key is to fit the tool to the problem in hand, 
bearing in mind that time and cost involved 
in gathering the data are important from a 
management perspective. This paper 
focuses on specific techniques appropriate 
for large research libraries. It illustrates a 
variety of methodologies, both formal and 
informal, with which we have had first 
hand experience. Cases have been drawn 
from technical, public, and administrative 
service areas of libraries. Because Total 
Quality Management emphasizes the use of 
quantitative data, we have focused on 
survey research with some of its attendant 
pitfalls. We have also included a 
discussion of the use of qualitative data, 
such as client interviews and focus groups, 
with particular attention to the problem of 
gathering data that is representative of the 
larger group. 

INTRODUCTION 

When we talk about gatliering client 
data, most of us automatically think in 
terms of surveys or interviews to figure out 
what our clients want or think about our 
services. Our intention is to go beyond this 
automatic assumption and explore some of 
the broader themes involved in gathering 
data, pointing out the good and bad points 



of various methods as well as ways to 
improve the validity of research. 

There has always been a split in the 
social sciences between quantitative and 
qualitative research. All too often this has 
been depicted as "scientific" versus 
"unscientific" research, "hard" versus 
"soft" data, and in other terms that really 
get in the way of taking a look at the 
strengths and weaknesses of different 
approaches. It is fair to say that 
quantitative and qualitative research 
methods are based on truly different 
imderlying assumptions. 

QUANTITATIVE VS. QUALITATIVE 
RESEARCH 

A quantitative approach is associated 
with objectivity, an understanding of 
probability, methodological rigor (i.e., you're 
collecting data concerning the same 
variables each time you conduct a survey — 
not asking each person a different set of 
questions), and makes substantial use of 
statistics (i.e., allows you to generalize to 
varying degrees). On the other hand, 
qualitative research is more subjective in 
nature, the methodologies incorporate a high 
degree of flexibility, and it makes use of 
language and description in providing 
insight into attitudes and behavior. 
Qualitative research can provide great 
depth and rmderstanding in a particular 
context. 

SOURCES OF DATA - PRIMARY AND 
SECONDARY 

There are three ways of gathering data. 
We can gather data through surveys (which 
include interviews), or we can observe 
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behavior, or we can create an experiment. 
Most data gathered in libraries has been 
through the use of surveys, but some 
research has been observational (for 
example, some of the most useful 
information dealing with how people use 
online catalogs has been observational in 
nature). For a variety of reasons (among 
them, lack of familiarity with experimental 
design and our inexperience with thinking 
about how experiments could even be useful 
in libraries), experiments are rarely done in 
libraries. 

All three of these techniques are ways to 
gather what is called primary data — the 
data that someone has collected through use 
of a survey, an experiment, or some sort of 
observation to meet a specific set of 
objectives. There is a second t3q?e of data 
out there known as secondary data. 
Secondary data are data collected by 
someone else for some (hopefully) known 
purpose, which may also be of use to us in 
understanding our clients. Our libraries are 
full of data that other people have 
collected, and our colleagues are sitting on 
data that often applies directly to our own 
concerns. Data collected in the past are 
often capable of being used more than once, 
to answer additional questions. 

For example, if we were interested in 
how satisfied our customers were with our 
online catalog, and want to focus on in 
improving the catalog, we would probably 
design some sort of questiormaire, or we 
might intervif w a set of library users. We 
would collect the data, write it up, and 
consider the project done. But because 
we're not used to thinking about using more 
than one technique at a time to collect data 
for a project, we don't think about whether 
we would get a better answer to our 
questions if we looked around at what 
might have been done earlier at our own 
institution or at a similar institution. We 
might be missing some useful clues that 



either better explain the data we've 
collected or indicate that we've missed the 
boat. 

In our hypothetical example where we're 
looking at the online catalog, surveys or 
interviews might yield data that looked 
perfectly reasonable and v;e might draw 
certain conclusions about what most 
concerns our users from that data. But 
what would you .*ind out, and how might 
your conclusions change if an institution 
with a similar catalog had conducted an 
observational study fairly recently? What if 
they had examined search logs for their 
online catalog and noticed a high number of 
searches for information done in the wrong 
database, or noticed a high number of 
poorly constructed searches, or conversely a 
great deal of sophisticated use of systems? 
Would you at least consider whether some 
of that data might be relevant to your 
research, that there might be enough 
similarities between the two situations to 
permit some use of the secondary data to 
explain some of your findings? Some of this 
is changing because the notion of 
benchmarking is encouraging us to look 
around at who else has information that 
may relate to our own problems. 
Benchmarking may have some of a ripple 
effect in that we'll find that information 
other than direct numerical comparisons can 
be highly relevant. 

WHY DO SURVEY RESEARCH? 

Researchers like surveys because they 
are not too intrusive, they are direct, they 
are usually anonymous so people may tell 
you things they wouldn't say to your face, 
the analysis is fairly straightforward, and if 
you've got a representative sample you can 
draw generalizations from your findings. 

What's wrong with surveys in general is 
that it's so easy to do them really badly. 

Bad survey research has contaminated 
things. All too often someone does a survey 
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with very little thought given to sampling 
techniques, the questions are poorly worded 
or biased, results are misinterpreted — i.e., 
they're done by people with little training in 
research in the social sciences. In addition, 
surveys are overused and perceived as 
nuisances, thanks to direct marketers with 
their suppertime phone surveys. 

TYPES OF SURVEYS 

Each type of survey also has its own set 
of advantages and disadvantages. What 
kinds of surveys are there? 

1. On the Spot Paper Questionnaires We 
Ask People to Fill Out. This type could 
even include some forms that we routinely 
require people to fill out, such as interlibrary 
loan forms. With little or no modification, 
you might be able to gather information 
from a real variety of places by using these 
existing forms. 

Advantages: 

a) very convenient 

b) can get a high response rate, captive 
audience 

Disadvantages: 

a) may not be able to get a very 
representative sample (look at all the 
research done based on college students) 

b) may not be able to do this in many 
situations 

c) often has to ba short, questions may be 
simplistic 

d) setting/ timing may not be ideal 

2. Mail Surveys. 

Advantages: 

a) convenient 

b) cheap 

c) potentially can get a very large group 
Disadvantages: 

a) poor response rate, can't generalize (a 
good response rate would be 50-70% 
with follow-ups) 



3. Phone Surveys. 

Advantages: 

a) fast: one week or less 

b) wide range 

c) goot' response rate (can achieve a 60-80% 
response rate with trained interviewers 
who are good on the phone) 

Disadvantages: 

a) can be very expensive 

4. In Person Interviews. 

Advantages: 

a) 80% plus response rate 

b) can be very cheap if you use volvmteers 
Disadvantages: 

a) can be very expensive if you use a 
consulting firm or professional 
interviewers 

b) interviewer bias 

c) interviewer effects (respondents may be 
embarrassed to make some comments) 

d) leading questions 

e) inconsistency (an interviewer may not 
always ask people the same questions) 

HOW TO DO SURVEY RESEARCH WELL 
There are eight basic questions to be 
answered when you do a survey research 
project. 

1. What Do You Want to Know? This is 
actually the most important decision. Don't 
expect a survey to answer all of your 
research questions. Keep the survey 
focused. 

2. Who Do You Need to Interview? In 
most cases, it is much more important to 
have a representative sample than a 
random sample. Figure out who knows 
what you want to know. 

3. How Are You Going to Select People to 
Interview? Aim for random selection. 
Avoid any sort of systematic bias. 
Remember that representation is the most 
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important factor. 

4. How Many People Do You Need to 
Interview? There are two things to 
remember. First, a sample's accuracy is 
more important than its size. In other 
words, are you interviewing the right 
people? Second, sample size is usually 
independent of population size. That is, 
just because you have a huge population 
don't assume that you need a huge sample. 
For example, election polls are based on a 
small sample but are representative of the 
population as a whole. 

5. What Are the Questions that Will Be 
Asked? Look at other surveys for good 
questions. Get advice from your colleagues. 
Try for multiple choice questions as much as 
possible but include some open-ended 
questions. Pretest your survey to eliminate 
ambiguous questions and avoid bad 
questions (See Figure 1). 

6. How Will the Questionnaire Be 
Administered? Ask for volimteers. You'll 
be building support for your project. If 
others feel that they have a stake in it, it's 
more likely to be successful. 

7. How Will the Data Be Analyzed? Get 
some advice from experts on your campus. 
Many spreadsheet packages will do a very 
basic analysis. Q&A or Excel could be used 
for frequencies. It is probably wise to build 
some expertise in SAS or SPSS. You can 
usually have a consultant or campus expert 
help with this part. 

8. What Do the Results Mean? Know the 
limits of your statistical analysis. Don't 
overgeneralize. Remember that in the social 
sciences the best we can usually do is 
describe what's going on, causality is 
extremely difficult to attribute. . 



EXAMPLES OF LIBRARY-BASED SURVEYS 

Examples of library-based surveys are 
appended (See Figures 2-4). Our survey 
examples include a traditional paper and 
electronic questionnaire, and a telephone 
survey. 

WHAT IS QUALITATIVE RESEARCH? 

The purpose of qualitative research is to 
uncover perceptions, feelings, attitudes, and 
ideas that relevant individuals have about 
an issue. It involves talking in depth with a 
few individuals or small groups. 

Qualitative research complements 
quantitative research. It should be used as 
an adjimct to quantitative research. It is not 
a replacement for quantitative research. 

It can be used in preparing for 
quantitative research. For example, 
qualitative research can be used to form a 
general h 3 qjothesis. It can also be used after 
quantitative research has been conducted to 
provide insight into survey results. 
Qualitative research can be particularly 
valuable for needs assessment and problem 
analysis. 

WHAT ARE SOME BASIC QUALITATIVE 
RESEARCH METHODS? 

Qualitative research uses two basic 
methods of data collection: 1) Individual 
interviews and 2) focus groups. 

With the first technique, carefully 
selected individuals are interviewed one at 
a time. Individual interviews are both 
structured and in-depth. 

Focus groups are a prevalent form of 
qualitative research. Instead of interviewing 
people one at a time, a focus group brings 
together a small group of people for a 
focused discussion. By definition, a focus 
group generally consists of 8 to 12 
participants from a target group who 
participate in a 90 to 120 minute structured 
discussion led by a trained moderator. 
Focus groups have been used for decades in 
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marketing and advertising research. Focus 
groups were used as early as the 1930's 
when women were asked about soap. The 
basic techniques were established in The 
Focused Interview, a 1956 classic by 
sociologist Robert K. Merton and others. 
Focus groups were originally perceived as a 
generic research method that could be 
applied to any setting. 

WHAT ARE THE STRENGTHS OF FOCUS 
GROUPS? 

This methodology goes beyond a 
questionnaire. It provides more personal, 
in-depth information than quantitative 
research. The purpose is to get small groups 
to articulate their beliefs about an issue. 

You can use focus groups to determine what 
your clients like and dislike. Focus groups 
can help explain people's behaviors. 
Individual comments are important because 
participants may make telling comments. 
Participants tend to be less inhibited in a 
group discussion than in individual 
interviews. The security of a group setting 
makes ttxem more likely to express their 
feelings. Using a group widens the range of 
responses. One individual's comments can 
trigger a chain of reactions from the other 
participants — i.e., one comment can have a 
snowballing effect. Focus group discussions 
can be more candid and spontaneous than 
individual interviews. Focus groups 
provide an opporhmity to probe your 
clients needs. By improving your 
understanding of clients needs, you become 
better able to improve the quality of services 
to clients. You can also use employee focus 
groups to solicit insights of employees into 
problems and solutions. 

Focus groups can be less time consuming 
than a written survey. They are also less 
time consuming than multiple individual 
interviews. If you are using individual 
interviews as a technique, you would 
probably have to conduct 15 to 30 



interviews in order to have an appropriate 
sample size. You can plan, conduct, and 
analyze the results of a focus group 
discussion in just a few weeks. 

In contrast to written surveys, focus 
groups are relatively inexpensive. The cost 
depends largely on the number of sessions 
you conduct, the choice of moderator, and 
the availability of facilities. You should use 
more than one focus group in case the 
responses of one group are atypical. A rule 
of thumb is to hold two to four groups per 
target audience. Ideally, you should 
continue to hold sessions until the 
discussion fails to reveal any new issues. 
Libraries are fortunate because they can use 
faculty members or graduate students w'ho 
have had experience with focus groups as 
moderators. In addition, facilities are 
generally free since the sessions can be 
conducted in ttxe library or elsewhere on 
campus. 

Good public relations can be an added 
benefit of focus groups. Participants often 
enjoy ttxe experience. It makes them feel 
that ttxeir opinions are important. 

LIMITATIONS OF FOCUS GROUPS 

You have to remember that focus group 
samples are both small and 
unrepresentative. You have to be careful 
not to project the results to a larger 
population since focus groups don't provide 
a large enough sample to allow for 
extrapolation. Since the results are not 
projectable, qualitative research is not a 
substitute for projectable quantitative 
research. Focus groups complement 
traditional surveys. 

You have to be cautious since intent and 
actions are two ''eparate things. 

Participants might respond that they would 
use the library more if hours were 
expanded. This doesn't mean that they 
would actually do so if hours were 
increased. 
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Group dynamics can impact the 
outcome. This is especially true if one or 
two participants monopolize the 
conversation. 

This is why having a goou moderator is 
the key. Success depends heavily on the 
moderator. A moderator has to be able to 
use a range of verbal and nonverbal 
techniques in order to ensure that every 
participant has an opportunity to speak 
and that no one monopolizes the 
discussion. A moderator needs to be able 
to direct but not control the discussion. 
Moderating is a skill. 

Focus groups have been widely used in 
colleges and universities. For example, 
admissions offices have used focus groups 
to determine how and why prospective 
students choose a college. Focus groups 
have only been used by libraries in the past 
decade. 

LIBRARIES CAN USE FOCUS GROUPS TO: 

1. Determine how well existing services 
meet clients needs. The Iowa City Public 
Library used focus groups to find out how 
the library could improve services to the 
business community. Individual interviews 
can also be used to find out what your 
customers think about service. 

Perm State recently conducted 16 
individual interviews with frequent and 
infrequent ILL users. First, an interviewer 
conducted in-depth, rmstructured 
telephone interviews with nine frequent ILL 
users (both faculty and graduate students) 
to determine what they thought about the 
service and their suggestions for 
improvement. Overall, the respondents 
were very positive about ILL and they did 
offer a number of suggestions for how 
service could be improved, such as the 
ability to send requests electronically. Next, 
the same interviewer conducted in-depth 
telephone interviews with seven infrequent 



ILL users. The interviewer asked four open 
ended questions: 1) What do you expect 
from ILL? 2) How' do you prefer to 
communicate with ILL? 3) What might 
encourage you to make more use of ILL?, 
and 4) What amount of time do you think 
is reasonable to wait for materials? The 
interviewer fotmd that this group tended to 
have an apathetic or negative view of ILL 
since they infrequently used the service and 
didn't know much about the service. In 
contrast, the first group tended to have 
more realistic expectations. 

2. Help design quantitative studies. 
Qualitative research helps you identify the 
questions to be asked. In addition to 
helping with the focus of a survey, 
qualitative research can help with the 
v/ording of the survey. You can use the 
language that participants use in order to 
increase the likelihood that survey 
respondents will imderstand the questions 
being asked. The individual interviews that 
Perm State conducted with ILL users were 
followed up by a print handout survey. 

3. Supplement data collected through a 
user survey. Purdue University used focus 
groups to gather student users opinions 
about the quality of service (i.e., the 
collection, staff, facility). These focus 
groups supplemented data gathered through 
a survey on student and faculty attitudes 
toward the library. 

4. Provide insight into non-use. While 
quantitative research provides data about 
who uses library services, qualitative 
research, particularly focus groups, provide 
insight into why people use or do not use 
library services. The Denver Public Library 
used focus groups to find out the needs of 
minority populations. This insight into non- 
use is critical if libraries want to improve 
services, increase use, and increase users 
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satisfaction. 

5. Plan for a new facility. Penn State is in 
the process of plarming for a major 
exparrsion that will result in several new 
libraries within a library, such as a business 
library, social science library, education . 
library, etc. Last year we conducted focus 
groups with faculty and students in order to 
gather input about the design of each of 
these specialized libraries. Although our 
intentions were good, we found that things 
could go wrong. One problem is that groups 
can be hard to assemble (especially faculty). 
No one showed up for one of the 
discussions and one of the other sessions 
had poor attendance. These sessions were 
scheduled in winter and bad weather forced 
us to reschedule one of the other meeting 
times. 

6, To solicit input on electronic resources. 
Cornell University used focus groups to find 
out what features and enhancements users 
would like to see incorporated into the 
Mann Library Gateway. 

A SUCCESSFUL FOCUS GROUP 
DISCUSSION REQUIRES: 

1. Established research goals. Define 
what the purposes of the study are. 
Determine what information is needed. The 
purpose of a focus group is not to arrive at 
a group consensus, change attitudes, or 
resolve conflicts, but to collect qualitative 
data to answer research questions. 

2. An identifiable target audience. 
Determine whom to study. Who should the 
participants be? Do you want to study 
clients or potential clients? If you are 
studying users of a particular service, such 
as ILL, are you interested in those who 
make limited, moderate, or heavy use of 
ILL? Participants should be carefully 



selected and should represent the 
individuals you want to study. 

3. An objective, enthusiastic, and 
knowledgeable moderator. An outside 
moderator is more objective. This is why 
it's best if the moderator is not a library 
staff member. It is important that the 
moderator be perceived as friendly. The 
moderator also needs to be knowledgeable 
about the subject. A moderator needs to be 
a good listener and needs to be able to draw 
out reticent participants. Good moderators 
ask probing questions such as "Give me an 
example of . . ." or "Would you explain 
further?" It is also important that the 
moderator try to riuniinize bias. An 
example of moderator bias would be the 
reinforcement of favorable comments by 
head nodding or praise such as "excellent," 
"wonderful," etc. 

4. A carefully planned discussion. The 
moderator, in conjunction with a library 
staff member, should prepare a written 
guide which outlines topics to be covered. 
This guide should have plarmed questions 
and a sequence of questions. A moderator 
should avoid questions that can be 
answered by a simple yes or no. A 
moderator should limit the number of topics 
to be addressed so each one can receive 
sufficient attention. A focused discussion 
generally revolves around fewer than ten 
questions. There are often only five to six 
questions with follow-up questions. If the 
topic is complex, there should be even fewer 
questions. Good focus group research is 
narrow in scope. Rather than gathering 
superficial information about many ideas, it 
is better to gather in-depth information 
about a few issues. Before each session, the 
moderator should welcome the group, 
explain how participants have been 
selected, identify the objectives, and 
establish basic ground rules, such as the 
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need for participants to speak one at a time 
and that there are no right or wrong 
answers. 

5. A record of session. The session can be 
recorded in several ways. First, you can 
have a representative from the library sit in, 
observe the session, and take notes. This 
observer does not participate. You should 
limit the number of observers so their 
presence is not intrusive. Another option is 
to have the observer behind a two-way 
nurror. Sessions are generally taped with an 
audio recorder. If the session is being 
taped, however, it is important that the 
moderator tell participants that they are 
being recorded. You can also consider using 
a moderator and an assistant r loderator. 
The moderator leads the discussion and 
takes minimal notes. The ass stant 
moderator takes extensive notes, operates 
any equipment, and handles logistics such 
as refreshments. After the session, the 
moderator transcribes the tapes. The 
moderator should prepare a written 
analysis soon after all the sessions have 
been conducted using the transcript. The 
moderator's notes and observer's notes can 
supplement the transcript if the tape is 
unclear. This written report should outline 
the broad themes which have emerged in 
several different sessions. There are 
commercially available computer packages 
(Qualpro, H}p>erqual) to analyze 
transcripts. These packages typically 
identify the frequency of words and 
phrases. The problem is that packages like 
these may take comments out of their 
context. 

6. A comfortable facility. The room you 
use should have corrrfortable chairs. It 
should be convenient, easy to find, and free 
from distractions. You should also serv’e 
refreshments. 



ASSESSING QUALITATIVE RESEARCH 
The following criteria can be applied to 
both individual interviews and focus 
groups. 

• Are the objectives clear? 

• Are the number of individual 
interviews and/or focus group 
discussions adequate for the issues 
being explored? 

• Have the participants been carefully 
selected? 

• Is the interviewer or moderator well 
trained? 

• Is there a written guide listing topics 
to be covered and a sequence of 
questions? 

• Does the analysis identify broad 
themes that have emerged in sessions? 

CONCLUSION 

Focus groups can be used in conjimction 
with individual interviews and surveys. 

The best approach to data collection is one 
that uses several methodologies. Both 
methods of gathering data (quantitative and 
qualitative) are useful and the relationship 
between them should be complimentary 
rather than adversarial. None of the 
techniques which we have presented are 
particularly complex. The key to success is 
developing good judgement about what 
techniques to use imder which 
drcximstances. 
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